Determination and use of link performance measures

ABSTRACT

Methods and apparatus for determining and using link performance measures (D L , C L ) in respect of communication links ( 125 ) in a cata communication network ( 100 ) are disclosed, where the network comprises a plurality of data-forwarding nodes ( 120 ) each capable of forwarding data along a plurality of communication links ( 125 ). Link performance measures (D L , C L ) are determined in dependence on end-to-end path performance measures (Es,u) received in respect of a plurality of end-to-end paths having at least one link in common, and on the basis of route information (Rs,u) identifying the links of which those end-to-end paths are comprised. Methods are disclosed for using link performance measures in relation to decisions regarding the routing and/or re-routing of data in a network, and in relation to determining measures indicative of expected quality of experience in respect of data traversing end-to-end paths across a network.

This application is the U.S. national phase of International ApplicationNo. PCT/GB2014/000337 filed 29 Aug. 2014 which designated the U.S. andclaims priority to EP Application No. 13250099.2 filed 30 Aug. 2013, theentire contents of each of which are hereby incorporated by reference.

TECHNICAL FIELD

The present invention relates to methods and apparatus for determininglink performance measures in respect of communication links in a datacommunication network and using such link performance measures inrelation to decisions regarding routing and/or re-routing of data in thenetwork, where the network comprises a plurality of data-forwardingnodes each capable of forwarding data along a plurality of communicationlinks. Further aspects of the present invention relate to methods offorwarding data in accordance with routing rules determined independence on such link performance measures, and to the use of suchlink performance measures in relation to determining measures indicativeof expected quality of experience in respect of data traversingend-to-end paths across a network.

BACKGROUND TO THE INVENTION AND PRIOR ART

There are various scenarios in which the end-to-end performance ofservices delivered across a network may be important, and various waysin which good or poor end-to-end performance may manifest itself and bemonitored or perceived by end-users. Various factors may affectend-to-end performance, including the performance of an end-user's ownlocal network or a computing device therein, but unsatisfactoryend-to-end performance in respect of a service is often caused by linkson the route that carries the service across a network, the choice ofwhich is generally not under the control of the end-user.

If the links making up a route currently being used for a service cannotsustain the service-level agreement (SLA) of the service, end-users ofthe service are likely to perceive that they have experienced poorperformance. Poor performance can manifest itself, for example, asdrop-outs in a video service, as poor voice-quality in a Voice-over-.Internet Protocol (VoIP) call, as slow response behaviour in anapplication running on a remote server, etc. Measuring the SLAparameters (e.g. delay, jitter, packet loss etc.) of a service may notbe sufficient to get a good idea about the end-to-end performance. Forexample, video content with rapid scene changes generally requires ahigher bit-rate to achieve the same quality of perception as videocontent with little movement. Therefore, small fluctuations in Qualityof Service (QoS) may not be noticeable to end-users in terms of Qualityof Experience (QoE), meaning that a service could still continue on aroute that occasionally shows degraded performance. In some scenarios,SLA parameters may be too lenient for good QoE, possibly having beenselected mainly out of cost considerations. Real-time collection ofper-link performance metrics from the network might be a challenge. Inother scenarios, many services may be transmitted over one link, thetheoretical link capacity of which should be able to sustain all of theservices, but fluctuating data rates may result in them competing witheach other for capacity at certain times, leading to a poor end-to-endperformance for some or all of them.

One or more links on a route may thus be insufficient for deliveringgood end-to-end performance of a service. In some scenarios, a networkoperator may have access to network performance metrics per link inreal-time, and may use these, but such real-time “per link” networkperformance metrics are not always available, and even if they are, theymight be spurious or poorly understood.

If routing is done according to standard protocols based simply onminimising hop-count, it may not be possible to pick links that canguarantee an SLA.

A paper entitled “QoE Content Distribution Network for CloudArchitecture” by Hai Anh Tran, Abdelhamid Mellouk and Said Hoceini(First International Symposium on Network Cloud Computing andApplications (NCCA), November 2011) relates to “cloud” services andtheir increasing use in the provision of network services. Due to thehigh bandwidth requirements of cloud services, use may be made ofContent Distribution Networks (CDNs), which may support high requestvolume and improve network quality using a mechanism based onreplication of information among multiple servers. The paper proposes aContent Distribution Network Cloud Architecture, which based not just onQuality of Service criteria (such as round trip time, network hops, lossrate, etc.) but also on the Quality of Experience that representsend-users perception and satisfaction. It describes how QoE scores maybe used in combination with QoS parameters to compute a link score orlink cost that can be used in a routing function, describing how QoEvalues can be sent back along the route data packets have travelled andhow link scores may be updates via a known method called “Q-Learning”.

Past techniques for predicting link failures and link QoS degradationhave generally required real-time link performance metrics, such asthose from a Management Information Base (MIB) of routers, to determinethese weak links.

International application WO 2012/085498 relates to communicationsnetwork management, and in particular to a communications network whichis divided into a plurality of segments, each segment comprising one ormore routers and one or more communications links that connect therouters. QoS thresholds can be defined for each of the segments, and ifit is predicted that one of these thresholds is to be breached in one ofthe segments, for example due to a communications link or a router beingoverloaded, then a segment management module associated with thatsegment can re-route the traffic.

International application WO 2012/085519 also relates to communicationsnetwork management and to a communications network which is divided intoa plurality of segments, each segment comprising one or more routers andone or more communications links that connect the routers. In this, eachsegment also comprises a segment management module. Each of the segmentmanagement modules reports to a supervisory management module (of whichthe network may have more than one). If a segment management modulepredicts that a QoS threshold will be breached, it may re-route a dataflow within that segment. If such a re-route is not possible, a requestmay be sent to the appropriate supervisory management module to initiatea re-routing to a further segment.

International application WO 2011/117570498 relates to a technique fornetwork routing adaptation based on failure prediction, and inparticular to a system that predicts network events and triggers apre-emptive response, and aims to predict network link failures andcreate a change in the network before the failure actually happens byinstigating policy-based adjustment of routing parameters. An exampleimplementation operates in two phases. In the first phase, thehistorical operation of a network is observed, to determine observedrelationships between link or cluster failures that have occurred, andsubsequent failures of different links or clusters. From these observedrelationships, failure rules can be derived that are then applied tocontrol routing in the network during a control phase. In this, thederived failure rules are applied such that if a link or cluster failureoccurs, then from the rules a prior knowledge of what additional linksmay fail in the next time period is obtained, and remedial action can betaken such as routing data traffic away from the links that arepredicted to fail.

In relation to techniques such as the above that predict link failuresand link QoS degradation, such that traffic can be re-routed aroundunderperforming link or such that session admission decisions can bemade, it has generally been assumed that MIB parameters are available inreal-time to the decision-making unit. Such parameters may not beavailable, however, or may be difficult to obtain and/or keep“concurrent” (i.e. updated appropriately by all instances). Forinstance, reported metrics can be out-of-date and/or not synchronisedwith reports from neighbouring routers (due to possible randomness inreport generation as well as lags in polling or delays/errors incurredin the network while transmitting such traps to the decision-makingunit). It might also generate management traffic loads from allintermediate routers that the operator might find undesirable.

Referring to other citations, United States patent applicationUS2008/080376 (“Adhikari”) relates to techniques for determininglocations or other causes associated with performance problems or otherconditions in a network, which may be used in network monitoring andanalysis systems for the monitoring and analysis of Voice over InternetProtocol (VoIP) communications, multimedia communications or other typesof network traffic.

US2006/274760 (“Loher”) relates to techniques for monitoring packetquality over an IP-based network by identifying sets of nodes andderiving the existence of the links between these nodes. The combinationof these nodes and links logically make up network paths. Qualitymeasurements performed across the network path can then be attributed tolinks, nodes, routes, networks, and other components of a communicationnetwork.

Japanese patent application JP2007221424 (“NEC”) relates to techniquesfor measuring communication quality in which end-to-end performances aremeasured on a number of routes and the minimum value of these isattributed to a link shared by the paths.

SUMMARY OF THE INVENTION

The present inventors have realised that issues such as those set outabove may be dealt with using a different approach that is not relianton information such as MIB parameters being available in real-time to adecision-making unit, and which is not reliant on link performancemetrics being available from intermediate nodes on a multi-link path.

According to a first aspect of the present invention, there is provideda method of deriving or updating routing rules in dependence on whichrouting decisions may be implemented by data-forwarding nodes in a datacommunication network, the data communication network comprising aplurality of data-forwarding nodes each capable of forwarding data alonga plurality of communication links, the method comprising

-   -   receiving end-to-end path performance measures in respect of a        plurality of end-to-end paths across the network, the end-to-end        paths each comprising a plurality of links which together form a        path for data to traverse the network from a data-sending        network-node to a data-receiving network-node, the or each        end-to-end path performance measure in respect of a particular        end-to-end path being dependent on and indicative of a network        performance metric observed at the data-sending network node of        that end-to-end path or at the data-receiving network-node of        that end-to-end path;    -   determining, in dependence on end-to-end path performance        measures received in respect of a plurality of end-to-end paths        having at least one link, in common, and on the basis of route        information identifying the links of which those end-to-end        paths are comprised, one or more link performance measures, the        or each link performance measure relating to a link of which at        least one of those end-to-end paths is comprised; and    -   deriving or updating routing rules in dependence on the one or        more link performance measures so-determined.

According to preferred embodiments, the route information may beobtained from a route information database. Alternatively oradditionally, the route information may be obtained from data units thatare intended to traverse, are traversing, or have traversed a pathacross the network.

According to preferred embodiments, the step of receiving end-to-endpath performance measures may comprise receiving end-to-end pathperformance measures from one or more data-sending network nodes and/orfrom one or more data-receiving network-nodes.

According to preferred embodiments, the received end-to-end pathperformance measures may comprise objective performance measures made inrespect of characteristics indicative of network performance on theend-to-end path. Alternatively or additionally, the received end-to-endpath performance measures may comprise subjective performance measuresmade in respect of characteristics indicative of network performance onthe end-to-end path.

According to preferred embodiments, the data traversing the network mayhave one of a plurality of categories associated therewith, and the stepof determining one or more link performance measures may compriseidentifying end-to-end path performance measures received in respect ofdata of one or more categories that is traversing the network, anddetermining, in dependence on end-to-end path performance measuresreceived in respect of data of said one or more categories traversing aplurality of end-to-end paths having at least one link in common, and onthe basis of route information identifying the links of which thoseend-to-end paths are comprised, one or more category-specific linkperformance measures, the or each category-specific link performancemeasure relating to performance in respect of said one or morecategories of a link of which at least one of those end-to-end paths iscomprised.

In such embodiments, the categories with which data traversing thenetwork are associated may include one or more categories such as classof service, type of service, or others.

According to preferred embodiments, the data-forwarding nodes are nodessuch as routers, capable of implementing routing decisions whereby toforward data units via any of a plurality of links. Such routingdecisions may be made by processors at the nodes themselves according topredetermined rules, based on information received about the performanceof the network and/or of links and other nodes, for example, or may bemade by one or more other entities and communicated to the nodes inorder to be implemented by them.

Methods according to preferred embodiments do not require real-time linkperformance metrics, such as those from the Management Information Base(MIB) of routers.

Methods according to preferred embodiments may be used by networkoperators for example, who may use them to identify poorly performinglinks in a network, for example. This may be done reactively and/orpredictively, with traffic being re-routed around links found to haveperformed or be performing poorly, or around links predicted to performpoorly (in the future generally, or at specific times of day, forexample).

Methods according to preferred embodiments may be used for theidentification of one or more links having a link performance measureindicative of performance below a predetermined threshold, in which caserouting rules may then be updated allowing routing decisions to be takensuch as to route data along other links, rather than those deemed to bepoorly-performing or “weak” links. Alternative embodiments may be usedfor the identification of one or more links, having a link performancemeasure indicative of performance above a predetermined threshold, inwhich case routing rules may then be updated allowing routing decisionsto be taken such as to route data along those deemed to behigh-performing or “strong” links, rather than other links.

Methods according to preferred embodiments may determine linkperformance measures based on end-to-end route performance measures suchas perception scores, which may also be used to determine measures ofoverall quality of experience. Where the end-to-end path is from adata-sending server to a data-receiving client, such end-to-endperception scores may be received from the client-side. Where the datais video content, for example, end-to-end perception scores indicativeof user-rated perceived video quality may be used. By mapping thevarious routes used by services on top of each other, weak links may bedetermined.

Using such techniques, the data that is required and the load on thenetwork due to management traffic may be greatly reduced.

Such techniques may be applicable in scenarios where real-timeperformance metrics may be inaccessible for constant retrieval but canbe collected in batches (say, overnight) instead. In such scenarios,prior art techniques may be inapplicable as they may require access tolink performance metrics to make real-time routing decisions.

Methods according to preferred embodiments may be used to identify linksthat are responsible for poor end-to-end QoE without directly measuringindividual link performance and relating it to an end-to-end performancemeasure.

A network management system using techniques such as the above mayidentify weak links over time that are responsible for poor end-to-endservice performance and ensure that they are less likely to be selectedor are not selected for routes in the future. This can lead to routechoices that better support a desired end-to-end service performancelevel.

By virtue of such a routing rule method, link performance measuresdetermined using methods according to the first aspect may be used toderive or update routing rules in dependence on which routing decisionsmay be implemented by data-forwarding nodes in a network. The routingrules may be for use by processors at the nodes themselves, for example,or may be for use by one or more other entities, with routing decisionsmade using those rules being communicated to the nodes in order to beimplemented by them.

According to a second aspect, there is provided a method of forwardingdata, the method comprising deriving or updating one or more routingrules using a method according to the first aspect, then forwarding datain accordance with said one or more routing rules.

According to a third aspect, there is provided a method of determining ameasure indicative of expected quality of experience in respect of datatraversing a particular end-to-end path across a network, the particularend-to-end path comprising a plurality of communication links whichtogether form a path via which data may traverse the network from adata-sending network-node to a data-receiving network-node, the methodcomprising, in respect of each of a plurality of links of which saidparticular end-to-end path is comprised, determining a link performancemeasure by:

-   -   receiving end-to-end path performance measures in respect of a        plurality of monitored end-to-end paths across the network, the        monitored end-to-end paths each comprising a plurality of links        which together form a path for data to traverse the network from        a data-sending network-node to a data-receiving network-node,        the respective end-to-end path performance measures in respect        of each monitored end-to-end path being dependent on and        indicative of a network performance metric observed at the        data-sending network node of that end-to-end path or at the        data-receiving network-node of that end-to-end path; and    -   determining, in dependence on end-to-end path performance        measures received in respect of a plurality of monitored        end-to-end paths having at least one link in common, and on the        basis of route information identifying the links of which those        end-to-end paths are comprised, one or more link performance        measures, the or each link performance measure relating to a        link of which at least one of those end-to-end paths is        comprised;    -   then determining a measure indicative of expected quality of        experience in respect of data traversing said particular        end-to-end path in dependence on the respective link performance        measures determined in respect of the plurality of monitored        end-to-end paths.

By virtue of such a method, link performance measures determined usingmethods according to the first aspect may be used to determine measuresindicative of expected quality of experience in respect of datatraversing an end-to-end path across a network. Such measures indicativeof expected quality of experience may themselves be used to make routingdecisions, for example.

According to a fourth aspect, there is provided apparatus for derivingor updating routing rules in dependence on which routing decisions maybe implemented by data-forwarding nodes in a data communication network,the data communication network comprising a plurality of data-forwardingnodes each capable of forwarding data along a plurality of communicationlinks, the apparatus comprising:

-   -   a receiver arranged to receive end-to-end path performance        measures in respect of a plurality of end-to-end paths across        the network, the end-to-end paths each comprising a plurality of        links which together form a path for data to traverse the        network from a data-sending network-node to a data-receiving        network-node, the or each end-to-end path performance measure in        respect of a particular end-to-end path being dependent on and        indicative of a network performance metric observed at the        data-sending network node of that end-to-end path or at the        data-receiving network-node of that end-to-end path; and    -   a processing module operable to perform steps of:        -   determining, in dependence on end-to-end path performance            measures received in respect of a plurality of end-to-end            paths having at least one link in common, and on the basis            of route information identifying the links of which those            end-to-end paths are comprised, one or more link performance            measures, the or each link performance measure relating to a            link of which at least one of those end-to-end paths is            comprised; and    -   deriving or updating routing rules in dependence on the one or        more link performance measures so-determined.

The various options and preferred embodiments referred to above inrelation to the first aspect are also applicable in relation to thesecond, third and fourth aspects.

BRIEF DESCRIPTION OF THE DRAWINGS

A preferred embodiment of the present invention will now be describedwith reference to the appended drawings, in which:

FIG. 1 is a system diagram illustrating the principal types of entitieswhich may be involved in performing route analysis according toembodiments of the invention;

FIG. 2 shows an example network with thirteen links and nine routes;

FIG. 3 is a flow-chart illustrating an example process for updatingvalues of diagnostic measures for links in a network;

FIG. 4 is a flow-chart illustrating an example process for updatingvalues of cost measures for links in a network;

FIG. 5 is a flow-chart illustrating the determination and use of linkperformance measures according to embodiments of the invention; and

FIG. 6 is a flow-chart illustrating how routing algorithms may use linkperformance or cost measures, both predictive and reactive, according toembodiments of the invention.

DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION

With reference to the accompanying figures, methods and apparatus fordetermining and using link performance measures according to preferredembodiments will be described.

FIG. 1 is a system diagram illustrating the principal types of entitieswhich may be involved in performing route analysis according to apreferred embodiment. It shows a network 100, which may be the Internet,a core network, or a part thereof, for example. It shows three servicehosts 110 (each labelled “H”), which are connected to (and are thus ableto send data to and/or receive data from) three end-user clients 130(each labelled “U”) via the network 100. The network 100 is shown ashaving three intermediate forwarding nodes or routers 120 (each labelled“N”) connected by network links 125.

Only a few service hosts 110 and end-user clients 130 are shown in FIG.1, and for simplicity the network 100 is shown with only a fewforwarding nodes or routers 120 and a few links 125. FIG. 2, which willbe discussed in more detail later, illustrates how even with only a fewentities involved, there may be several routes for the provision ofnetworked services by hosts to clients, and that these routes may sharesome or all of the links of which they are comprised with other routes.It will be understood that the aim of FIG. 1 is mainly to indicate thetypes of entities which may be involved, and that a core network wouldin general have many more routers and links that are shown in FIGS. 1and 2. It would in general be capable of facilitating data communicationbetween much greater numbers of network nodes (such as service hosts110, end-user clients 130, and others) along a much greater number ofpossible paths between data senders and data receivers. It will also beappreciated that the characterisation of entities as service hosts 110sending data or end-user clients 130 receiving data is purely forsimplicity—the entities which communicate via such a network may simplybe service providers or service users, and may simply be senders of dataor receivers of data, but they may fulfil more than one such role.Whether the entities communicating via a network are(primarily) actingas senders of data or receivers of data, or are performing bothfunctions, the edge nodes 110,130 via which they are connected to thecore network 100 may be referred to generally as end-user network nodes.

FIG. 1 also shows a link performance analysis system 10. This has aninput 12, an analysis module 14 and an output 16. The input is incommunication (via the network 100, via another network, orotherwise—possible communication lines are indicated by dotted lines)with some or all of the end-user network nodes 110,130, and is therebyable to receive information from them, including path performancemeasures (as will be discussed in more detail later). In someembodiments, it may receive path performance measures only fromdata-receiving end-user network nodes (such as end-user clients 130),while in others, it may receive path performance measures only fromdata-sending end-user network nodes (such as service hosts 110), but ingeneral, it may receive path performance measures from data-receivingand data-sending end-user network nodes.

As will be discussed in more detail later, path performance measuresreceived at the input 12 of the link performance analysis system 10 arepassed to an analysis module 14, which has access to a route database18. The route database 18 may be a part of the link performance analysissystem 10, or it may be an external database with which the analysismodule 14 of the link performance analysis system 10 may communicate(via the network 100, via another network, or otherwise). It is able toprovide route information in respect of particular end-to-end pathsbetween particular data-sending and data-receiving end-user networknodes, and/or in respect of particular network services provided byparticular data-sending nodes to particular data-receiving nodes, theroute information identifying the links of which those end-to-end pathsare comprised. The route information in the route database may beupdated in response to information received about faults, linkperformance, topology changes, and other factors which may affect theroute that data may take across the network.

It will be noted that in alternative embodiments, such route informationneed not be provided from a single dedicated route database. Dependingon the entities concerned, the type of data being exchanged, thecommunication protocols being used and other factors, the routeinformation may instead be obtained from a distributed route database,from network nodes (possibly via Link State Advertisement (LSA) or othermessages), from data units that are intended to traverse, aretraversing, or have traversed a path across the network 100, or in otherways. Such information may thus be provided to the system 10 by the dataunits in question or others, or by network nodes that have sent,forwarded or received data units, the route information being provideddirectly or via one or more route databases, for example.

The analysis module 14 is arranged to process the received pathperformance measures together with route information, and to determine,in dependence on path performance measures received in respect ofend-to-end paths having at least one link in common together with routeinformation identifying the links of which those end-to-end paths arecomprised, link performance measures, the link performance measuresrelating to links of which those end-to-end paths are comprised.

Link performance measures determined by the analysis unit 14 are thenprovided to an output 16 of the link performance analysis system 10,which may provide them to a unit 20 such as a route manager, to allowthe link performance measures to be taken into account in furtherrouting decisions, and/or to a diagnosis unit for further diagnosticanalysis. The route manager and/or diagnosis unit 20 may be a part ofthe system 10, or may be separate, and may be distributed orcentralised.

In embodiments in which link performance measures determined by thesystem 10 are to be taken into account in further routing decisions,information concerning the link performance measures, updated routingtables or routing decisions may need to be provided to network nodessuch as data-sending network nodes (service hosts 110, for example),intermediate network nodes (forwarding nodes or routers 120, forexample) or other network entities such as nodes holding routingdatabases. Connectivity back into the network for such purposes is notshown in FIG. 1 in order to avoid this becoming unnecessarily cluttered,but it will be understood that there are several possible options fordoing this.

Diagnostic Usage

For the purposes of describing a preferred embodiment, we will considera scenario in which a network (which in general would have many morenodes than network 100 as shown in FIG. 1) carries a multitude ofend-to-end services. These services can include video streams, VOIPcalls, interactive applications, multiplayer games, etc.

Intelligent analysis in respect of path performance measures received inrespect of the passage of data units across the network is performed bya link performance analysis system 10, which will generally be referredto as a route analyser 10. According to this embodiment, the primarymeasure that is computed by the route analyser 10 in respect of each ofa plurality of links in the network order to identify weak links is alink performance measure D_(L). This is a diagnostic measure for a givenlink L, which in this example will be taken to be in a range from 0to 1. A D_(L) value of 1 means that the given link is only part ofroutes that currently have optimal end-to-end performance (orperformance above a threshold regarded as “good” or “acceptable”, forexample). A D_(L) value of 0 means that the link is part of one or moreroutes that are suffering from poor end-to-end performance.

As will be apparent, link performance measures so-determined may beprovided to data senders and/or to routers and other network nodesresponsible for forwarding data, to a unit such as a route manager whichmay use them in order to determine how to route or re-route data(communicating its decisions to senders and forwarding nodes asapplicable), and/or to a diagnosis unit for further analysis, forexample. As will be explained in more detail later, link performancemeasures determined according to some embodiments may be usedpredictively, rather than for immediate or direct use in routing orre-routing, however.

Computation of D_(L) Values for Links in a Network

Briefly, D_(L) may be calculated in the following manner, a detailedprocess for which will be described later with reference to FIG. 3.

According to the present embodiment, two types of input are used byroute analyser 10 to compute D_(L) for the links in the network:

1) A first type of input indicating, in respect of particular end-to-endpaths, the quality of experience observed by an end-user in respect ofdata travelling to or from them via the path in question; and

2) A second type of input indicating, in respect of particularend-to-end paths, individual links making up the route taken, beingtaken, or to be taken across the network from a particular ingress pointto a particular egress point by data forming part of a particular flowor service.

In this example, inputs of the first type are computed or otherwisedetermined on the client side, and indicate a measure of the overallquality of experience from the point of view of data-receiving end-userclients 130. This measure can be purely objective, or can take intoaccount subjective factors that reflect the perception of the end-user.Measures can be, for example, subjective quality of perception for videostreams, pixilation of videos, number of dropped frames in the contentper time interval, voice quality of VOIP calls, and applicationperformance like end-to-end response time. Note that these are allexamples of end-to-end performance measures, subjective or otherwise.Purely subjective measures can be, for example, user feedback on a scaleof 1 to 10, 1 meaning poor and 10 meaning excellent.

It will be appreciated that the end-to-end performance measures may bedetermined or computed in a variety of different ways. In this example,however the measure is obtained, it will be assumed that it provides anumber which can be normalised such that 0 represents poor (or otherwiseunacceptable) performance and 1 represents excellent (or otherwiseacceptable) performance. This end-to-end quality measure will henceforthbe referred to as E_(S,U) for a given service S, reported by an end-userclient U that consumes this service.

The clients 130 regularly or frequently report their E_(S,U) measure ormeasures to the route analyser 10. They may communicate directly withthe route analyser.

It will be understood that in alternative embodiments, the E_(S,U)measures may reflect performance from the point of view of the servicehosts 110 as well as or instead of performance from the point of view ofthe clients 130. If it is desired that the measure should take intoaccount performance from the point of view of service hosts 110 andclients 130, to make the measure dependent on parameters observed byboth, both may communicate directly with the route analyser 10.Alternatively, for example, the clients 130 may feed their data back tothe service hosts 110, who may include their data into a combinedmeasure before providing this to the route analyser 10. This may allowservice hosts providing a service to multiple clients to determine aperformance measure in respect of multiple clients. The service host maycombine the values received from clients on a “per client” and/or on a“per time period” basis. A suitable summarisation would be, for example,to compute averages per client every 5 minutes and to store the averagesfor a moving 24 hour time window. The reporting entity, be it theservice hosts or the clients, would in general decide upon anaggregation method and reporting interval. For example, reporting of anaverage or minimum E_(S,U) could be done at the end of a service tocharacterise the performance for the duration of the service.

In the following, for simplicity, we consider the situation where theend-to-end performance values are determined by service hosts 110 (whichmay be with or without information received from their clients 130), whoprovide the E_(S,U) values to the route analyser 10.

The route analyser 10 maintains a table with the following information,based on information received from service hosts H1, H2 and H3:

TABLE 1 Table T_(E-2-E) of end-to-end performance values maintained by aservice host Service Application Class of End-User host Type ServiceClient Time E_(S, U) H1 Video stream EF U1 23/11/2012 10.55 0.67 H1Video stream EF U1 23/11/2012 11.00 0.56 H2 Database AF U2 23/11/201208.00 0.98 Access H3 Video stream EF U3 23/11/2012 08.00 0.34 . . . . .. . . . . . .

For the second type of input, the route analyser 10 receives thehop-by-hop route of the service S in question within the network fromthe ingress point to the egress point. This can be obtained in a numberof ways, depending on the protocol stack used by the application. Forexample, it can be obtained from resource reservation protocols such asRSVP. In a network that operates a link-state protocol such as OSPF, theend-to-end route within the network can be obtained from the protocolitself as it supports a topology map of the network and associated linkcosts. Alternatively, if fixed MPLS tunnels have been configured, theseroutes can be known in advance. If multicast traffic traverses thisnetwork, routing information for the multicast trees can be obtainedfrom the respective routers' Tree Information Base (TIB). We refer toroute information for a given service S to a particular end-user clientU as R_(S,U).

E_(S,U) and R_(S,U) data is consumed by the route analyser 10. Thisentity can rest anywhere in the network including at ingress routers, atservice hosts or as a dedicated unit. The route analyser 10 cancorrelate the T_(E-2-E) tables with the route information for theservices shown in the table to compute average E_(S,U) values per route.Alternatively, average E_(S,U) values can be computed per route (i.e.for each Service Host: End-User Client combination), per class ofservice/type of service/time of day, or sub-combinations thereof.

In this example, D_(L) is computed as follows. If a link is part of aroute, it is assigned the E_(S,U) value reported for this route. If thelink is part of only one route, this becomes its D_(L) value. If thelink is part of more than one route, its D_(L) value is the average ofthe E_(S,U) scores of all of these routes. For example, if link L. ispart of three different routes, the services on which have E_(S,U)values of 0.3, 0.7 and 0.8 respectively, the D_(L) value for link L is0.6. Links in respect of which no E_(S,U) value or route information isknown may have their D_(L) value initialised to 1.0 by default.Evidently this is a trusting approach where links are not negativelyimpacted if there is no information about them. Alternative approachesare also possible where, for example, an average or pre-determined D_(L)value is assigned if no information is available from the link L. D_(L)values can also be grouped by application type or Class of Service(“CoS”), resulting in a vector of D_(L) values per link. The averagecalculated over the number of routes supported by L can be computed as aweighted average to differentiate in importance between the routes andthe services they support. The route E_(S,U) value can also be weightedby the number of clients consuming the service(s) on this route.Additionally, the update of a D_(L) value can be done as a smoothedaverage over time, combining new information with the old average, toavoid rapid changes.

The D_(L) values may be updated periodically, e.g. every 5 minutes, toreflect changes in end-to-end performance of routes. If a link is nolonger part of any routes, information about it may not be reported byany clients. In view of this, D_(L) values may be “aged” appropriately.This may ensure that links that are not used slowly return to D_(L)=1.0again to avoid being penalised (or to an average value, or to some otherdefault value, as explained above).

It will be noted that a network may support various classes of service.In such scenarios, it may be appropriate to incorporate informationabout these into the computation of D_(L) in different ways, either toproduce individual D_(L) values for the different classes, or to producecombined D_(L) values reflecting the overall performance of links forthe data of the various classes they are carrying. As mentioned above,it would be possible to have a vector of D_(L) values for the multipleclasses of service. Using the score for routing purposes (as will bediscussed later) then means that the network may also have an awarenessof available traffic bandwidth in each class of service on a link L toavoid sending a high bandwidth of traffic through the link based on ahigh D_(L) value which might result in a see-saw effect of decreasingthe D_(L) value due to subsequent congestion. This approach results in aseparate link model across the network being developed for each class ofservice. Whilst the granularity of this approach is an advantage, thisapproach can be quite complex. Alternatively, one other possibleimplementation is to combine the vector of D_(L) values into a singlevalue by giving weighting factors to each of the classes of service andusing this to compute a single D_(L) value. Factors to take into accountin assigning these weighting factors for link L could include: bandwidthavailability for the CoS in the link L (related to queue managementpolicies in its associated routers) and importance of that particularCoS to the operator. For example, EF/AF traffic (i.e. “ExpeditedForwarding” and “Assured Forwarding”) could be considered to be moreimportant than DE traffic (“Default”).

A more detailed description of how D_(L) values may be determined willnow be given with reference to FIG. 3. It will be noted that while thisis a possible method to compute D_(L) values, this method can bereplaced by a variety of others, subject to the necessary networkknowledge being available. It will also be noted that other types oflink performance measures may be determined instead. A detaileddescription of how another such type of link performance measure may bedetermined will be given later with reference to FIG. 4.

Referring to FIG. 3, an algorithm to update the D_(L) values for alllinks may be invoked periodically by the route analyser 10 to updateD_(L) values in respect of some or all of the links 125 in the network100. It begins in step s300 and first runs a loop (s302-s306) wherecounters temp_L and count_L are initialised to zero (s304) for each linkL in the current link set LS. The route analyser maintains a link set LSwhere it stores all links it has encountered in any route and theirrespective D_(L) values. If the link set LS has been exhausted in s306the algorithm continues with two nested loops s308-s324. The outer loops308-s324 runs over all entries in the table T_(E-2-E). The inner loops312-s322 runs over all links L in a particular route R. In s308 theroute analyser retrieves the next entry from T_(E-2-E) and identifiesservice host H and end-user client U in respect thereof. In s310 theroute R from H to U is determined based on information from the routedatabase. The route R consists of a list of links, and the inner loops312-s322 involves running through all links in the route to add up thescores from the T_(E-2-E) table and to count the number of services inwhich a link is used. In s314 it is established if the link is alreadycontained in the current link set LS. If a link has not been seen by theroute analyser, it gets a default D_(L) value of 1, the link is added tothe link set LS in s316, and counters temp_L and count_L are initialisedto zero (s318). Otherwise the process continues directly with s320. Ins320 the E_(S,U) score is added to the current sum for L in temp_L andincrease L's service counter is increased by 1. The process continueswith the inner loop from s312 if the test in s322 reveals more links inthe route R, otherwise it is checked in s324 if there are moreunprocessed entries in T_(E-2-E). If so the process continues with theouter loop from s308, otherwise it continues with s326. From s326,another loop s326-s336 starts in which the new D_(L) values are computedfor each link in the link set LS. The route analyser takes the next linkfrom the link set LS in s326 and checks in s328 if the service counteris zero (to avoid divisions by zero). If the counter is not zero, D_(L)is computed according to the formula in s330 which is a weighted averageof the old D_(L) value and the new average score temp_L/count_L. Aweight “w” between 0 and 1 is used here. For example, setting w=0.9means that 90% of the current D_(L) are maintained, but this can bechosen in the light of how strongly current values are to be taken intoaccount in comparison with recent or older values. In the exampledescribed earlier, a weight w=0.5 is used. If the counter is zero thenD_(L) is not recomputed and it is checked in s332 if the D_(L) valueshould be aged. This decision can be based on some timer that providesinformation relating to the intervals over which D_(L) values shouldreturn to 1 for a link which is no longer part of any route (e.g. everyhour, every 24 hours, etc.). If the decision is made to age the D_(L)value, the formula in s334 is applied using the same weight w. Otherwisethe process jumps to the end of the loop s336, where it is tested ifmore links are present in the link set LS and have to be processed. Ifthere are more links left, the process continues with the loop froms328, otherwise the process for performing the algorithm completes ats338.

Exemplary pseudo-code which may be invoked periodically by a routeanalyser 10 according to an embodiment in order to update all D_(L)values in the network using a process such as that described above andshown in FIG. 3 could be as follows:

For each L in LS    temp_L = 0; count_L = 0; End For each entry inT_(E-2-E)    H = Service Host; U = End-User Client    R = route(S,U)   For each L in R      If not(L in LS)        then add L to LS; D_(L) =1; temp_L = 0; count_L = 0;      end      temp_L = temp_L + E_(S,U);     count_L = count_L + 1;    end End For each L in LS    if count_L ==0      then if time_to_age        then D_(L) = 1 − w + w* D_(L) /* ageD_(L)        if it is not in any route */      end      else D_(L) =D_(L) * w + (1-w) /* (temp_L/count_L); /*      update D_(L) */    EndEndUse of D₁ Values

D_(L) values determined as set out above or using processes according toother embodiments may be used simply to diagnose “weak” links in thenetwork. This may be achieved in the absence of real-time networkperformance metrics (from the MIB, for example). A low D_(L) value for alink L may indicate that the link is part of one or more underperformingroutes, whereas a high D_(L) value may indicate that the link is part ofone or more routes that support services with good QoE. Using anappropriate threshold, links with a consistently low D_(L) value can behighlighted to an operator for investigation and troubleshooting.

D_(L) values may also be used as a trigger or basis for re-routing. Thismay be achieved in real-time, the D_(L) values being used by the routeanalyser 10 or another entity to trigger re-routing around weak links.The method of pushing the routing changes back to the routing entities(to distributed routing daemons running on each of the routers, or to acentralised entity that handles local routing, for example) will not bediscussed in detail here, as well-known techniques may be used.Generally, however, a mechanism of communication would exist between theroute analyser 10 and the forwarding plane where packet switching takesplace. Methods of achieving this include using Telnet, or using an APIthat interfaces between the network and the intelligence unit.

Routing traffic away from an underperforming link may allow it thechance to recover, should congestion be the cause of poor performance.Eventually, while not being used, the D_(L) value of a link mayincrease, after which it may be autonomously reused in routing.

The following is an example of how D_(L) values can be used inconjunction with existing routing protocols.

A simple way to turn a D_(L) value into a link cost value (where D_(L)values are in a range from 0 to 1) is to compute 1-D_(L), in accordancewith routing protocols that are configured to prefer lower link costsover higher ones. This can either be used on its own or combined withthe existing routing metric. This can be done by weighting the D_(L)value in relation to the unicast (or multicast) routing cost and addingit to the existing cost metric. The weight applied to the two costsdepends on the importance that the operator wishes to apportion to eachof the two components.

From the above, three possible applications of D_(L) are evident: 1)determination of weak links for the operator; 2) reactive re-routingaround weak links based on D_(L) values; and 3) proactive usage of D_(L)to influence routing algorithms for subsequent routes.

As mentioned earlier, types of link performance measure other than D_(L)may be determined. A technique in respect of one such measure, a costmeasure C_(L), will now be described.

Computation of C_(L) Values for Links in a Network

We now describe an alternative measure that can be computed instead ofD_(L). We call this C_(L) and it is a cost value that reflects thenumber of times a given link L is part of poorly performing routes. Thisis different to D_(L) since the latter is a function of perceivedend-to-end performance, whereas C_(L) is a function of the number oftimes a link is part of any route perceived as being an underperformingroute (thus not needing to take into account the extent of the poorperformance). The C_(L) value of a link L essentially reflects thenumber of occasions that the link is common to two or moreunderperforming routes.

Briefly, C_(L) may be calculated in the following manner, a detailedprocess for which will be described later with reference to FIG. 4. Theroute analyser 10 first sorts the routes into high-performing andlow-performing routes based on E_(S,U) values. The route analyser 10maintains a list of poorly performing routes (as determined from theE_(S,U) value) and removes routes from this list if they are no longerin use or if their E_(S,U) values subsequently increase. Using thehop-by-hop route information described earlier, it identifies commonlinks in this list of poorly performing routes. It can do this bycomputing the length “le” of a route and adding a value of 1/le to theC_(L) value of a link L. Length le is the number of hops the routecontains from one client to another. The C_(L) value is a penaltymeasure for poor performance of a route and by apportioning it equallyto each component link in the route, the penalty can be distributedevenly. This is because knowledge that an end-to-end route is exhibitingpoor performance may not allow a determination as to which of itscomponent links may be responsible for the overall poor performance.This procedure may be performed for all links and all the routes in thenetwork segment. Therefore, if a link is a common link of several poorlyperforming routes, its link cost C_(L) increases through this procedure(as will be described with reference to FIG. 4) with each additionweighted by the length of the route that contributes to that increase.If a link is shared by many poorly performing routes, its C_(L) valuewill be higher than that of other links.

If information about network congestion is known, this can also be usedto increment C_(L) values accordingly by incrementing C_(L) by a valuerelated to congestion instead of 1/le.

C_(L) values are initialised to a default value, which could be setdepending upon the class of service that this C_(L) value is computedfor, so that links can be differentiated when used in routing algorithmsper class of service.

The route analyser may age C_(L) values and decrease them over time suchthat link costs return to their default values if links are no longerused or no longer appear as a common link in low performing routes.

It will be understood that there could be a higher number of levels intowhich route performance may be classified, such as “high”, “medium” and“low”, of course. Similarly, a three-tier classification system for theperformance of individual links may be used. In such a case, the networkcould continue using the component links classified to be of “medium”performance under certain (less congested) network conditions, forexample. Alternatively, a link that is part of a “medium” performingroute may be penalised by a different value added to its C_(L) factor.Such a system can be expanded, each tier of classification being treateddifferently and contributing differently to the cumulative C_(L) valuefor link L, for example.

A process for determining C_(L) values (in a case where the end-to-endquality measure E_(S,U) classifies route performance according to twolevels, indicating end-to-end performance above or below a predeterminedthreshold) will now be described with reference to FIG. 4.

Referring to FIG. 4, an algorithm to update the C_(L) values for alllinks may be invoked periodically by the route analyser 10 to updateC_(L) values in respect of some or all of the links 125 in the network100. It begins in step s400 and first runs a loop (s402-s406) where acounter ctemp_L is initialised to zero (s404) for each link L in thecurrent link set LS_C, ctemp_L being a temporary variable. The routeanalyser maintains a link set LS_C where it stores all links it hasencountered in any route and their respective C_(L) values. If the linkset LS_C has been exhausted in s406 the algorithm continues with twonested loops s408-s424. The outer loop s408-s424 runs over all entriesin the table T_(E-2-E). The inner loop s412-s422 runs over all links Lin a particular route R. In s408 the route analyser retrieves the nextentry from T_(E-2-E) that has an E_(S,U) score below a chosen threshold“t” (e.g. t=0.4, the level of which determines the level below which aroute should be classed as poorly performing) and identifies servicehost H and end-user client U in respect thereof. In s410 the route Rfrom H to U is determined based on information from the route database,and its length le (which may be measured as a number of hops) isestablished. The route R consists of a list of links, and the inner loops412-s422 involves running through all links in the route to add acontribution to the ctemp_L count that reflects the chance that aparticular link is responsible for having caused the E_(S,U) score forthe route it is in to have been below the threshold “t”. In s414 it isestablished if the link is already contained in the current link setLS_C. If a link has not been seen by the route analyser, it gets adefault C_(L) value of 0, the link is added to the link set LS_C ins416, and counter ctemp_L is initialised to zero (s418). Otherwise theprocess continues directly with s420. In s420, the current sum for L isincremented in counter ctemp_L by an amount inversely proportional tothe number of hops in the route. The process continues with the innerloop from s412 if the test in s422 reveals more links in the route R,otherwise it is checked in s424 if there are more unprocessed entries inT_(E-2-E). If so the process continues with the outer loop from s408,otherwise it continues with s426. From s426, another loop s426-s436starts in which the new C_(L) values are computed for each link in thelink set LS_C. The route analyser takes the next link from the link setLS_C in s426 and checks in s428 if the service counter is zero (to avoiddivisions by zero). If the counter is not zero, C_(L) is computedaccording to the formula in s430 which is a weighted average of the oldC_(L) value and the new ctemp_L count. A weight “v” between 0 and 1 isused here, similarly to the weight “w” used in the process of FIG. 3. Ifthe counter is zero then C_(L) is not recomputed and it is checked ins432 if the C_(L) value should be aged (as explained in relation to FIG.3, for example). If the decision is made to age the C_(L) value, theformula in s434 is applied using the same weight v. Otherwise theprocess jumps to the end of the loop s436, where it is tested if morelinks are present in the link set LS_C and have to be processed. Ifthere are more links left, the process continues with the loop froms428, otherwise the process for performing the algorithm completes ats438.

Exemplary pseudo-code which may be invoked periodically by a routeanalyser 10 according to an embodiment in order to update all C_(L)values in the network using a process such as that described above andshown in FIG. 4 could be as follows:

For each L in LS_C    ctemp_L = 0; End For each entry in T_(E-2-E) withE_(S,U) < t    H = Service Host; U = End-User Client    R = route(S,U);   le = length(R);    For each L in R      If not(L in LS_C)        thenadd L to LS_C; ctemp_L = 0; C_(L) = 0;      end      ctemp_L = ctemp_L +1/le;    End End For each L in LS_C    if ctemp_L > 0      then C_(L)=v * C_(L) + (1-v) * ctemp_L;      else if time_to_age        then C_(L)= v * C_(L)      end    end End(NB The function route(S,U) determines all links on the current route Rfrom service host H to end-user client U. The function length(R) returnsthe number of links/hops on the route.)Use of C_(L) Values

C_(L) values may be used similarly to D_(L) values as described above(e.g. for diagnostics and for re-routing). C_(L) values can also bedirectly used in cost calculations without needing to subtract them from1 in order to obtain a cost value (as described above in relation toD_(L)). C_(L) values may need to be scaled to be comparable to othercost metrics used by existing routing algorithms, however, depending onhow much importance the operator wishes to apportion to C_(L) comparedto existing link cost measures.

It will be noted that C_(L) values, like D_(L) values, generally becomemore reliable when more routes are analysed and several consistent routeoverlaps through common links are found. Using too few routes mightresult in erroneous penalising of common links between these routeswhere congestion might actually have occurred elsewhere on the routesbut the common links are penalised instead. Analysing several routesincreases the likelihood of route overlaps and therefore improves theefficiency of diagnosis.

Predictive Usage

As indicated earlier, link performance measures determined according tosome embodiments may be used predictively, rather than for immediate ordirect use in routing or re-routing, however. The following discussionrelates to this aspect.

For predictive usage, a multilayer perceptron (or similar machinelearning or predictive data analytics algorithm) may be used. It will beunderstood that a multilayer perceptron is only one of many possiblemachine learning techniques that can be used, however.

In this example, the neural network comprises of an input layer, one ormore hidden layers and one output layer. (A detailed discussion of howto configure and train a multilayer perceptron with training data willnot be given here, as known techniques may be used.) The neural networklearns the relationship between a route and its end-to-end performanceto predict the latter, given the former.

Training the Neural Network

We explain here how the neural network may be built. Possibleapplications in prediction are described in the next section.

In this example, a route in the IP network is represented as a vector ofcomponent links. This is derived as explained in earlier sections (usingrouting protocols or RSVP). Each link is then represented for the neuralnetwork as a number of link performance metrics such as throughput,delay, jitter, MIB metrics, link cost including D_(L) or C_(L) values,etc. There should be at least one performance metric per link, and themetric set used should be consistent for all links, i.e. there should beno mix of different types of performance metrics for different links.

Links are represented in the input layer of the neural network by groupsof input neurons. Each input neuron group contains one input neuron foreach available link performance metric. The input layer of themultilayer perceptron contains enough groups of input neurons to acceptlink performance metrics for the longest possible route in the network.Shorter routes are padded by “perfect” links.

The purpose of the neural network is to predict the end-to-endperformance of a route (i.e. the E_(S,U) value) given a set ofperformance metrics for the component links of this route as well as anadditional optional set of parameters (described in the next paragraph).The output layer of the neural network will have one output neuron foreach type of end-to-end performance measure (E_(S,U)) that we want toconsider. This is applicable if we use, for example, a different E_(S,U)value for each service S provided by a service host H to an end-userclient U. Alternatively, we can use separate neural networks for eachE_(S,U) value. This could be done if a single neural network turns outto be too hard to train using the available data, i.e. if itsperformance would be too low after training, for example.

The training data used for the neural network are vectors containingthree groups of values:

-   -   1) n groups of k link performance values l_(j) ^((i)) (j^(th)        value for i^(th) link)    -   2) s optional parameters, p_(i), which can be used to describe        other relevant measures like time of day, amount of traffic on        the route, number of services on the route etc.    -   3) m E_(S,U) values, which we now call e_(i).        l_(j) ^((i)) and p_(i) are input values. e_(i) are the target        values.    -   (l₁ ⁽¹⁾, . . . , l_(k) ⁽¹⁾, l₁ ⁽²⁾, . . . , l_(k) ⁽²⁾, . . . ,        l₁ ^((n)), . . . , l_(k) ^((n)), p₁, . . . p_(s), e₁, . . . ,        e_(m))

If a route has fewer than n links, the missing link performance valuesare imputed by “perfect” link performance values, i.e. valuesrepresenting maximum possible throughput, zero latency, zero jitter,etc. These perfect values are considered to be network parameters andare set by the operator. The exact values do not matter as long as theyare genuinely describing optimal link performance in the network.

The data needed for building the neural network is collected by theroute analyser 10. It uses information from the T_(E-2-E) tabledescribed earlier plus additional link and network performance metricsthat are available from the network 100. Network performance metrics canbe obtained from Simple Network Management Protocol (SNMP) agents,resource reservation protocols such as Resource Reservation Protocol(RSVP), media control protocols such as RTP Control Protocol (RTCP) andInterior Gateway Protocol (IGP)/Multiprotocol Label Switching(MPLS)/multicast routing tables.

The training database will be constantly refreshed with new data and olddata will be removed. For example, records older than 24 hours may bedeleted. The neural network may be regularly retrained, for example,every hour, to make sure it constantly reflects the current networkstructure and behaviour.

After training, the neural network may be applied if a routing algorithmwants to predict the end-to-end performance of a given route. A route ismapped into a vector of component links, which is then represented as asequence of input values for the neural network. The output of theneural network is the predicted end-to-end performance of the route,i.e. its predicted E_(S,U) value. We refer to this predicted E_(S,U)value as PE_(S,U). This can be one or more values depending on theconfiguration of the neural network and the nature of the end-to-endperformance measures used.

Application of the Neural Network

If a route is proposed by a routing algorithm for a given service, theroute analyser uses the neural network to predict its end-to-endperformance PE_(S,U) as described in the previous section. The neuralnetwork produces an output value between 0 and 1. The routing algorithmcan use the prediction to accept or discard a potential route or pickthe best from a number of possible routes. This is the effect that thelearning algorithm has on the network. Additionally, a predictive D_(L)value (PD_(L)) and a predictive C_(L) value (PC_(L)) can be computed asdescribed in the “Diagnostic Usage” section earlier. The reason thevalues are predictive is that they are based on PE_(S,U) values insteadof actual E_(S,U) values. Using such predictive performance indicators,the route analyser proactively determines links to avoid because theyare expected to perform poorly, and therefore allows poor end-to-endperformance to be avoided in the first place. This is different tore-routing using E_(S,U), D_(L) and C_(L) values, which indicate thatpoor QoE has already occurred—using these measures for routing (in itsmost proactive form) only serves to prevent future drops of QoE. Anon-predictive version may well run through cycles of poor and betterend-to-end performance, whereas a predictive system may prevent poorend-to-end performance entirely after its initial training phase.

Worked Example

Consider the example network in FIG. 2, discussed earlier. This has fourservice hosts 110 (labelled as H1 to H4) providing services to threeend-user clients 130 (U1 to U3) via a network which includes fiveintermediate forwarding nodes or routers 120 (N1 to N5) and thirteenlinks (L1 to L13), which provide nine possible end-to-end paths orroutes for the delivery of services to the end-user clients. The routesor paths used for the services are indicated by the numbers 1 to 9 indashed-line boxes. For example, path 1 goes from client H1 to router N1via link L1, which also carries data using path 7, as indicated by thedashed-line box containing the path numbers “1, 7”. Path 1 continues vialink L6 to router N2, then via link L7 to router N3, and on via link L9to end-user client U1, as indicated by dashed-line boxes on each of therelevant links containing the path number “1” (amongst other pathnumbers).

Service hosts H1-H4 maintain end-to-end performance tables with thefollowing information:

TABLE 2 Table T_(E-2-E) of end-to-end performance values maintained byvideo server H1 Class of End-User Service Service Client TimePerformance Video-stream EF U1 23/11/2012 11.00 0.77 Video-stream EF U223/11/2012 11.00 0.86 Video-stream EF U1 23/11/2012 18.00 0.98Video-stream EF U2 23/11/2012 18.00 0.84 . . . . . . . . . . . .

TABLE 3 Table T_(E-2-E) of end-to-end performance values maintained bydatabase server H2 Class of End-User Service Service Client TimePerformance Database AF U1 23/11/2012 11.00 0.58 Database AF U223/11/2012 11.00 0.74 Database AF U1 23/11/2012 18.00 0.67 Database AFU2 23/11/2012 18.10 0.59 . . . . . . . . . . . .

TABLE 4 Table T_(E-2-E) of end-to-end performance values maintained byvideo server H3 Class of End-User Service Service Client TimePerformance Video-stream EF U1 23/11/2012 20.00 0.77 Video-stream EF U223/11/2012 20.00 0.21 Video-stream EF U3 23/11/2012 20.00 0.19 . . . . .. . . . . . .

TABLE 5 Table T_(E-2-E) of end-to-end performance values maintained byapplication server H4 Class of End-User Service Service Client TimePerformance Application EF U2 23/11/2012 20.00 0.25 Application EF U323/11/2012 20.00 0.21 . . . . . . . . . . . .

In this example, H1 is a video server serving end-user clients U1 and U2via routes 1 and 7; H2 is a database server serving U1 and U2 via routes2 and 3; H3 is another video server providing a video stream to U1, U2and U3 via routes 4, 5 and 8; and H4 is an application server serving U2and U3 via routes 6 and 9.

We consider the computations for links L7 and L11. The computations forthe other links work accordingly. We assume that an ageing mechanismhappens only every 24 hours and is not relevant for this example. Everytime a link value is updated this is done by averaging the previousvalue with the newly computed value. The threshold “t” for a lowperforming route/link is 0.40.

At 11:00 the following computation happens for L7. Assume the previousvalue is D₇=0.78 and C₇=0D ₇=(0.78+((0.77+0.86+0.58+0.74)/4))/2=0.75875

C₇ remains 0

At 18.00 we compute for L₇D ₇=(0.75875+((0.98+0.84+0.67+0.59)/4))/2=0.764375

C₇ remains 0

At 20.00 we compute for L7 and L11 (assume the previous values for L11are D11=0.55 and C₁₁=0)D ₇=(0:764375+0.77)/2=0.7671875

C₇ remains 0D ₁₁=(0.55+((0.21+0.19+0.25+0.21)/4))/2=0.3825

Routes 5, 6, 8 and 9 are below the threshold of 0.40 which triggers theanalysis for weak links in the route analyser. The links used by theseroutes are L4, L5, L11, L12 and L13. The length/e of each of the routesis 3. That means we obtain the following C_(L) values:C₄=0.67, C₅=0.67, C₁₁=1.33, C₁₂=0.67 and C₁₃=0.67.

If service hosts H3 and H4 request re-routing due to low performance,this can result in a better route that does not use link L11, identifiedas the most likely cause of low end-to-end performance for the servicesoffered by H3 and H4 since it carries the largest C_(L) value. Forexample, the video session from H3 to U2 could be rerouted via L3, L7and L10, thus removing traffic from L11. For the sessions to U3 thereare no alternative routes available for H3 and H4. However, removingtraffic from L11 could result in a better performance.

If we assume that at 11.00 the traffic on route 1 is 25 Mb/s and that wehave the following physical performance values for the links of route 1:

-   -   L1: loss=0.1%, delay=10 ms, jitter=2 ms    -   L6: loss=0.09%, delay=13 ms, jitter=1.5 ms    -   L7: loss=0.1%, delay=12 ms, jitter=1 ms    -   L9: loss=0.1%, delay=10 ms, jitter=1 ms

If we also assume that the longest route is of length 4, then we obtainthe following training data vector for the neural network at 11.00 forroute 1:

(0.1, 10, 2, 0.09, 13, 1.5, 0.1, 12, 1, 0.1, 10, 1, 11, 0, 25, 0.77)

The first 12 values are link performance values, the next two values (11and 0) are the hour and minute of the performance measurement, the nextvalue is the traffic on the route and 0.77 is the reported end-to-endperformance. Other training vectors are formed accordingly.

Referring now to FIG. 5, this illustrates how link performance measuresmay be determined and used in various ways according to preferredembodiments. Steps s500 to s520 may be performed in a mannercorresponding to the steps shown in the flow-charts of FIG. 3 or 4 (orotherwise), so will not be discussed again in detail.

Starting from step s500, following receipt of path performance measures(such as the E_(S,U) measures referred to earlier in relation to FIGS. 3and 4) for multiple end-to-end paths (s505), route information (such asthe R_(S,U) data referred to earlier) is used (s510) to identify thelinks of which the end-to-end paths in question are comprised. Fromthis. it is possible to identify paths having links in common (s515),and thus to determine link performance measures (D_(L), C_(L) orotherwise) for those links (s520). Link performance measuresso-determined can then be used in order to derive or update routingrules (s525), which may be done as shown in FIG. 6 (see below) orotherwise. The process may then end (s530) or be repeated. In addition,however, such new or updated routing rules may then be provided (s540)to network nodes (120 in FIG. 1) requiring them in order to takedecisions as to how to forward data. These nodes may implement these newor updated rules in order to compute an appropriate route (s545), andthus to forward data on the computed route (s550). The process may thenend (s555) or be repeated.

Alternatively or additionally, link performance measures determined asset out above may be used for making predictions of expected quality ofexperience for particular end-to-end paths where, for example, nocurrent or recent end-to-end path data exists for those paths, but whereit is or has been possible to determine link performance measures forall (or at least most) of the links making up those paths. Such atechnique may involve identifying (at s560) one or more paths for whichcurrent or recent link performance measures are known for all (or most)links, then using the link performance measures for those links in orderto determine quality of experience predictions for the end-to-end pathsin question (s565). The process may then end (s570) or be repeated.

Referring now to FIG. 6, this illustrates how routes may be computedpredictively or reactively using routing algorithms that may use linkperformance measures or cost measures. Looking at this in more detail,routes for forwarding data may be determined using routing algorithmsthat use link performance measures D_(L), predictive link performancemeasures PD_(L), link cost values C_(L) or predictive link cost valuesPC_(L). If D_(L) or C_(L) values are used (i.e. reactive techniques),the route may be computed based on current measured end-to-endperformance and the current link performance (D_(L)) or cost (C_(L)). Ifthe route is to be computed based on predicted end-to-end performance(i.e. predictive techniques), then PD_(L) or PC_(L) values are used bythe routing algorithm.

Starting from step s600, following receipt of a routing request (s605),a first-stage decision is taken as to whether D_(L) values or C_(L)values are to be used (s610). A second-stage decision is then taken(s620 or s640) as to whether routes are to be computed predictively orreactively. Depending on the outcome of these stages, PD_(L) values(step s625), D_(L) values (step s630), PC_(L) values (step s645) orC_(L) values (step s650) are retrieved. In the case of PC_(L) values orC_(L) values, these may be used as they stand in order to compute aroute (s660). In the case of PD_(L) values or D_(L) values, these may beconverted to cost values (PC_(L) or C_(L)) before then being used tocompute a route (s660). The process may then end (s670) or be repeated.

The invention claimed is:
 1. A method of determining a measureindicative of expected quality of experience in respect of datatraversing a particular end-to-end path across a network, the particularend-to-end path comprising a plurality of communication links whichtogether form a path via which data may traverse the network from adata-sending network-node to a data-receiving network-node, the methodcomprising, in respect of each of a plurality of links of which saidparticular end-to-end path is comprised, determining a link performancemeasure by: receiving end-to-end path performance measures in respect ofa plurality of monitored end-to-end paths across the network, themonitored end-to-end paths each comprising a plurality of links whichtogether form a path for data to traverse the network from adata-sending network-node to a data-receiving network-node, therespective end-to-end path performance measures in respect of eachmonitored end-to-end path being dependent on and indicative of a networkperformance metric observed at the data-sending network node of thatend-to-end path or at the data-receiving network-node of that end-to-endpath; and determining, in dependence on end-to-end path performancemeasures received in respect of a plurality of monitored end-to-endpaths having at least one link in common, and on the basis of routeinformation identifying the links of which those end-to-end paths arecomprised, one or more link performance measures, each of the one ormore link performance measures relating to a link of which at least oneof those end-to-end paths is comprised; then: determining the measureindicative of expected quality of experience in respect of datatraversing said particular end-to-end path in dependence on therespective link performance measures determined in respect of theplurality of monitored end-to-end paths; wherein said receivingend-to-end path performance measures comprises receiving end-to-end pathperformance measures from one or more data-sending network nodes and/orfrom one or more data-receiving network-nodes; and wherein the routeinformation is obtained from data units that are intended to traverse,are traversing, or have traversed a path across the network.
 2. Themethod according to claim 1 wherein the route information is obtainedfrom a route information database.
 3. The method according to claim 1wherein the received end-to-end path performance measures compriseobjective performance measures made in respect of characteristicsindicative of network performance on the end-to-end path.
 4. The methodaccording to claim 1 wherein the received end-to-end path performancemeasures comprise subjective performance measures made in respect ofcharacteristics indicative of network performance on the end-to-endpath.
 5. The method according to claim 1 wherein the data traversing thenetwork has one of a plurality of categories associated therewith, andwherein the step of determining one or more link performance measurescomprises identifying end-to-end path performance measures received inrespect of data of one or more categories that is traversing thenetwork, and determining, in dependence on end-to-end path performancemeasures received in respect of data of said one or more categoriestraversing a plurality of end-to-end paths having at least one link incommon, and on the basis of route information identifying the links ofwhich those end-to-end paths are comprised, one or morecategory-specific link performance measures, the or eachcategory-specific link performance measure relating to performance inrespect of said one or more categories of a link of which at least oneof those end-to-end paths is comprised.
 6. The method according to claim5 wherein the categories, with which data traversing the network areassociated, relate to class of service categories or type of servicecategories.
 7. The method according to claim 1 wherein thedata-forwarding nodes are nodes capable of implementing routingdecisions whereby to forward data units via any of a plurality of links.8. An apparatus for determining a measure indicative of expected qualityof experience in respect of data traversing a particular end-to-end pathacross a network, the particular end-to-end path comprising a pluralityof communication links which together form a path via which data maytraverse the network from a data-sending network-node to adata-receiving network-node, the apparatus comprising: a receiverarranged to receive end-to-end path performance measures in respect of aplurality of monitored end-to-end paths across the network, themonitored end-to-end paths each comprising a plurality of links whichtogether form a path for data to traverse the network from adata-sending network-node to a data-receiving network-node, therespective end-to-end path performance measures in respect of eachmonitored end-to-end path being dependent on and indicative of a networkperformance metric observed at the data-sending network node of thatend-to-end path or at the data-receiving network-node of that end-to-endpath; and a processing module operable to determine a link performancemeasure in respect of each of a plurality of links of which saidparticular end-to-end path is comprised by determining, in dependence onend-to-end path performance measures received in respect of a pluralityof monitored end-to-end paths having at least one link in common, and onthe basis of route information identifying the links of which thoseend-to-end paths are comprised, one or more link performance measures,each of the one or more link performance measures relating to a link ofwhich at least one of those monitored end-to-end paths is comprised; theprocessing module further being operable to determine the measureindicative of expected quality of experience in respect of datatraversing said particular end-to-end path in dependence on therespective link performance measures determined in respect of theplurality of monitored end-to-end paths; and wherein the routeinformation is obtained from data units that are intended to traverse,are traversing, or have traversed a path across the network.